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May  17,  1990 


The  Honorable  Richard  B.  Cheney 
The  Secretary  of  Defense 

Dear  Mr.  Secretary: 


The  Congress  continues  to  express  concern  that  weapon  systems  have 
begun  production  without  operational  test  and  evaluation  (ot&e).  ot&e 
conducted  before  a  system’s  ‘‘production  start-up,”1  is  a  key  internal 
control  to  ensure  that  decisionmakers  have  the  best  information  availa¬ 
ble  on  a  weapon  system’s  performance  to  minimize  risks  of  procuring 
costly  and  ineffective  systems. 

We  identified  many  perceived  barriers  to  earlier  ot&e.  Several  had  been 
previously  reported  by  us  and  other  agencies,  and  others  were  identified 
during  discussions  with  officials  in  the  Office  of  the  Secretary  of 
Defense  (osd)  and  the  military  services.  This  report  provides  the  results 
of  our  analysis  of  the  feasibility  of  overcoming  three  major  barriers  that 
we  believe  will  allow  ot&e  before  production  start-up.  The  three  barriers 
were 

lack  of  prototypes  for  testing  early, 

starting  production  before  development  is  completed  (concurrency),  and 
the  need  to  start  production  to  obtain  ot&e  test  articles.2 

We  recognize  that  other  factors,  such  as  cost,  urgency,  and  the  state  of 
advanced  technology  may  sometimes  preclude  ot&e  before  production 
start-up.  However,  making  ot&e  results  available  before  production 
start-up  could  help  preclude  cost  growth,  schedule  slippages,  and  per¬ 
formance  shortfalls  that  frequently  arise  during  the  later  phases  of  a 
weapon  system’s  development. 

Several  events  make  the  present  systems  acquisition  environment  very 
receptive  to  earlier  ot&e.  These  include  (1)  recent  dod  reports  that 
espouse  the  ‘‘aggressive  use  of  prototyping  and  testing  to  identify  and 
remedy  problems  before  production,”  (2)  dod’s  revisions  to  acquisition 
and  test  and  evaluation  regulations  currently  underway,  (3)  the 


1  In  this  report,  we  use  the  term  production  start-up  to  refer  to  that  point  in  the  acquisition  process  at 
which  a  weapon  system  or  its  subsystems  begin  production.  "I/>w-rate  initial  production  IIJRIPI." 
"limited  production,”  and  "pilot  production"  are  terms  that  the  Department  of  Defense  (DOD)  uses  to 
refer  to  production  start-up. 

2Test  articles  are  the  units  of  a  weapon  system  that  are  used  to  test  a  weapon’s  performance 
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Results  in  Brief 


Background 


appointment  of  a  new  Director  of  ot&e,  (4)  the  need  to  make  critical 
trade-off  decisions  in  systems’  acquisition,  and  (5)  dod’s  agreement,  in 
response  to  our  May  1989  report  discussed  on  page  4,  to  reemphasize 
the  need  for  ot&e  as  early  as  possible  in  the  acquisition  cycle. 


Several  dod  and  congressional  initiatives  to  reduce  risks  in  system  acqui¬ 
sition  and  many  reports  have  highlighted  weapon  system  performance 
and  testing  shortfalls.  Our  review  of  six  current  systems  plus  other 
recent  wcrk  shows  that  dod  has  made  little  progress  in  assuring  that 
earlier  ot&e  is  planned  and  conducted.  The  military  services  generally 
are  not  conducting  or  planning  to  conduct  OT&E  on  weapon  systems  until 
after  production  start-up. 

In  our  view,  the  barriers  we  examined  do  not  preclude  earlier  ot&e.  With 
proper  planning,  an  operational  test  can  and  should  be  accomplished 
before  production  start-up.3  We  found  that  although  some  prototypes 
have  been  used  for  development  test  and  evaluation,  the  military  ser¬ 
vices  generally  are  not  planning  to  use  them  for  ot&e  before  production 
start-up.  We  believe,  however,  that  with  adequate  planning,  prototypes 
can  be  designated  for  ot&e.  We  also  found  that  concurrency  does  not 
preclude  earlier  ot&e,  if  such  testing  is  properly  planned.  Moreover, 
because  test  articles  are  available  during  the  full-scale  development 
phase  of  the  acquisition  process,  it  is  generally  not  necessary  to  begin 
production  to  provide  them  for  ot&e. 


dod  budgets  over  $100  billion  annually  for  research,  development,  test, 
and  evaluation  and  for  production  of  major  weapon  systems.  In  view  of 
such  large  sums,  decisionmakers  need  the  best  information  possible  on 
weapon  systems’  operational  performance  and  risks  associated  with 
new  technology  before  approving  production  start-up.  ot&e  is  intended 
to  provide  an  independent  analysis  of  a  weapon  system’s  performance 
against  operational  requirements  and  the  system’s  reliability  and  ability 
to  be  supported  in  the  field. 


’OT&E  before  production  start-up  may  be  in  addition  to  later  OT&E  conducted  to  support  a  full-rate 
production  decision  (often  referred  to  as  the  "final  exam").  OT&E  before  production  start-up  ( 1 )  may¬ 
be  on  a  system  or  critical  subsystem  prototype,  (2)  might  not  meet  all  requirements  for  effectiveness 
or  suitability,  and  (3)  should  not  be  viewed  as  a  "pass/fail"  test  but  can  serve  as  an  input  to  the 
design  process  to  ensure  problems  are  addressed  before  a  system  begins  production. 
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With  Proper  Planning, 
OT&E  Before 
Production  Start-Up  Is 
Feasible 


The  services  are  statutorily  required  to  conduct  ot&e  before  making  a 
full-rate  production  decision,  dod’s  current  test  and  evaluation  policy4 
not  only  reflects  this,  but  also  states  that  “ot&e  shall  be  structured  to 
provide  inputs  at  each  decision  point,  including  major  milestones.”5  Yet, 
dod’s  policy  also  permits  production  start-up  before  ot&e  is  conducted. 

Studies  of  the  dod  acquisition  process  have  recommended  earlier  opera¬ 
tional  testing,  using  prototypes,  before  production  start-up.  In  1986,  the 
President’s  Blue  Ribbon  Commission  on  Defense  Management  (the  Pack¬ 
ard  Commission)  recommended  that  operational  testing  of  prototype 
systems  begin  early  in  advanced  development  and  continue  through  full- 
scale  development,  dod’s  recent  Defense  Management  Review  initiatives, 
as  reflected  in  the  July  1989  dod  Report  to  the  President  on  Defense 
Management  and  the  January  1990  dod  Annual  Report  to  the  President 
and  the  Congress,  support  implementation  of  the  Packard  Commission 
recommendations,  including  aggressive  use  of  prototyping  and  testing  to 
identify  and  remedy  problems  “well  prior  to  commencement  of  high  rate 
production.” 


Over  the  years,  we  have  reported  on  weapon  system  performance  short¬ 
comings  and  the  lack  of  adequate  ot&e.  (See  app.  III.)  Our  work  on  six 
current  systems  plus  our  other  recent  work  (see  apps.  I  and  II)  has 
shown  that  ot&e  is  not  being  conducted  until  production  start-up  has 
been  approved.  We  understand  that  the  military  services  want  to  move 
weapon  systems  into  production  as  quickly  as  possible  in  order  to 
reduce  the  likelihood  of  them  being  canceled — a  “build  now  and  fix 
problems  1  at  ,  ’  attitude. 

We  believe  that  o,&E  results  have  not  been  available  to  support  decisions 
to  start  production  because  of  the  failure  of  the  military  services  to  plan 
for  earlier  ot&e  and  incorporate  it  into  program  acquisition  strategies. 
The  barriers  often  cited  for  ot&e  not  being  adequately  planned  for  or 
conducted  before  production  start-up  has  been  approved  include 
(1)  prototypes  not  available  for  testing  early,  (2)  concurrency,  and 
(3)  the  need  to  start  production  to  obtain  ot&e  test  articles.  We  believe 
that  these  barriers  do  not  preclude  ot&e  prior  to  production  start-up. 


'In  DOD  Directive  5000.3,  Test  and  Evaluation,  dated  March  12,  1986. 

The  milestones  are:  I  (to  start  the  concept  demonstration/  validation  phase),  II  (to  enter  the  full-scale 
development  phase),  and  III  (to  enter  the  production  phase). 
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Prototypes  Are  Being  Built 
but  Not  Being  Used  for 
CJT&E 


In  a  recent  report  on  test  and  evaluation,1'  we  discussed  the  manner  in 
which  dod  used  early  operational  assessments.  Early  operational  assess¬ 
ments  provide  input  to  decisionmakers  on  whether  a  weapon  system  is 
ready  for  the  “final  exam  ot&e”  and  whether  operational  shortfalls  are 
being  identified  and  corrected  early  in  the  development  process.  These 
assessments,  which  usually  rely  on  development  test  data,  are  being 
used  to  support  decisions  for  full-scale  development  and  production 
start-up.  We  reported  in  May  1989  that  dod’s  reliance  on  early  opera¬ 
tional  assessments  when  equipment  is  unavailable  for  or&E  is  a  step  for¬ 
ward  in  filling  the  information  void,  but  should  not  be  a  substitute  for 
ot&e.  However,  we  support  dod’s  current  policy  that  calls  for  ot&e  at 
each  decision  point,  including  major  milestones,  and  believe  that  more 
emphasis  should  be  given  to  conducting  ot&e  before  production  start-up. 
This  ot&e  should  not  be  viewed  as  a  pass/fail  test;  it  would  function  as 
an  input  to  decisionmakers  to  ensure  that  problems  are  identified  before 
a  system  begins  production. 


Our  review  has  shown  that  even  though  prototypes  have  been  built  or 
are  under  construction  for  some  systems,  the  services  generally  do  not 
plan  to  use  them  to  conduct  ot&e.  For  example,  prototypes  were  built  or 
under  construction  for  five  of  the  six  systems  we  reviewed  (the  Air 
Force’s  Advanced  Tactical  Fighter  and  Short  Range  Attack  Missile-II, 
the  Army’s  Non-Line-Of-Sight  Missile,  and  the  Navy’s  Long-Range  Air 
Anti-Submarine  Warfare  Capability  Aircraft  and  Fixed  Distributed  Sys¬ 
tem).  However,  only  the  Navy  used  or  has  plans  to  use  its  prototypes  in 
ot&e  before  production  start-up.  This  is  because  only  the  Navy  has  regu¬ 
lations  calling  for  ot&e  before  production  start-up.  The  other  services’ 
regulations  only  require  ot&e  to  support  full-rate  production  decisions, 
although  some  field  testing  or  experimentation  may  be  conducted. 

The  Navy  used  the  Fixed  Distributed  System  prototype  to  conduct  ot&e 
before  production  start-up.  Although  the  Fixed  Distributed  System  was 
not  a  mature  system  with  a  prototype  available  for  only  one  of  its  two 
major  subsystems,  the  Navy  considered  the  ot&e  results  to  be  useful. 
This  ot&e  was  conducted  before  full-scale  development. 

We  recognize  that  not  all  systems  can  or  should  have  prototypes.  How¬ 
ever,  ot&e  conducted  before  the  decision  to  begin  production  could  be 
performed  on  critical  subsystems  if  it  is  not  feasible  to  have  a  prototype 


'’Navy  Weapons  Testing:  Defense  Policy  on  Early  Operational  Testing  (GAONSI  AD-89-98.  Mav  8. 


Page  4 


GAO/NSIAD-90-107  OT&E 


B-222886 


of  the  full  system.  The  controlling  factors  would  be  system/subsystem 
maturity  and  the  expected  risk. 

For  earlier  ot&e  to  be  conducted,  acquisition  strategy  from  the  beginning 
r  should  require  that  prototypes  be  built  and  operationally  tested  before 

the  decision  to  begin  production  is  made.  This  is  the  responsibility  of 
acquisition  managers,  not  operational  testers. 


Concurrency  Does  Not 
Preclude  OT&E  Before 
Production 


As  we  have  previously  reported,7  concurrency  can  be  an  effective  tech¬ 
nique  for  expediting  acquisitions  if  it  is  well  planned  and  controlled; 
however,  the  practice  increases  the  risk  that  systems  will  be  produced 
with  major  flaws,  ot&e  before  production  start-up  is  an  especially  impor¬ 
tant  safeguard  against  the  increased  risks  of  concurrent  programs. 

On  April  17,  1990,  dod  reported  to  the  Congress  that  its  policy  allows 
varying  degrees  of  concurrency,  but  mandates  the  completion  of  testing 
and  the  assessments  of  results  before  the  production  phase.8  Proposed 
concurrency  guidelines  call  for  the  aggressive  use  of  prototypes  and 
testing  to  identify  problems  well  before  production.  Under  normal  cir¬ 
cumstances,  the  low-rate  initial  production  decision  should  be  supported 
by  the  completion  of  some  early  ot&e. 

Most  dod  officials  we  talked  with  concerning  earlier  ot&e  believe  that  it 
is  feasible  to  conduct  ot&e  before  most  production  start-up  decisions. 
However,  our  discussions  identified  several  factors  that  need  to  be  con¬ 
sidered  when  ot&e  is  to  be  conducted  before  the  production  start-up 
decision: 

ot&e  must  be  planned  as  part  of  the  system’s  acquisition  strategy. 
Systems  tested  early  may  not  meet  all  requirements  for  suitability  (i.e., 
reliability,  availability,  and  maintainability). 

Decisionmakers  need  to  be  told  that  ot&e  earlier  in  the  acquisition  pro¬ 
cess  should  not  be  viewed  as  pass/fail;  it  should  be  viewed  as  a  way  to 
identify  system  performance  shortfalls  that  need  to  be  corrected  before 
production  begins. 


7Production  of  Some  Major  Weapon  Systems  Began  With  Only  Limited  Operational  Test  and  Evalua¬ 
tion  Results  (GAO/NSi AD-85-68.  June  19,  1985). 

hAs  requested  by  section  801  of  the  National  Defense  Authorization  Act  for  Fiscal  Years  1990  and 
1991 ,  DOD  was  to  report  on  the  risk  associated  with  concurrency  in  major  acquisition  programs 


Page  S 


GAO/NS1AD-90-107  OT&E 


B-222886 


We  believe  that  these  factors  are  also  important  to  systems  that  are  not 
categorized  as  concurrent  programs. 


Production  Start-Up  Is 
Generally  Not  Necessary 
for  OT&E  Test  Articles 


Production  start-up  is  traditionally  considered  as  a  means  by  which  test 
articles  are  acquired  for  crr&E.  The  military  services  usually  use  the  term 
low-rate  initial  production  to  describe  this.  However,  the  Congress  and 
dod’s  Inspector  General  have  expressed  concern  that  the  services’  use  of 
low-rate  initial  production  has  sometimes  resulted  in  de  facto  full-rate 
production  before  any  crr&E  is  conducted.  To  try  to  control  the  misuse  of 
low-rate  initial  production,  in  November  1989,  the  Congress  passed  leg¬ 
islation  requiring  an  appropriate  dod  official  to  specify  the  quantities  of 
test  articles  needed  for  crr&E  in  major  programs  to  decrease  the  number 
of  items  produced  before  ot&e  was  conducted. 


We  found  that  production  start-up  is  generally  not  necessary  to  produce 
or&E  test  articles.  The  dod  budget  manual  and  the  military  services’  reg¬ 
ulations  require  or  at  least  allow  ot&e  test  articles  to  come  out  of  full- 
scale  development.  In  addition,  we  identified  many  systems  currently 
under  development  or  in  production  where  the  services  plan  to  use  or 
have  used  ot&e  test  articles  produced  from  full-scale  development.  (See 
app.  I.)  In  only  one  of  the  six  systems  we  reviewed  will  the  ot&e  test 
articles  come  from  a  low-rate  initial  production  decision.  In  that  case, 
according  to  the  Deputy  Program  Manager,  the  Line-of  Sight-Forward- 
Heavy  was  already  in  production  for  an  allied  country  before  the  U.S. 
Army  bought  it. 


Conclusions  and 
Recommendations 


Weapon  systems  are  starting  production  before  ot&e  is  conducted 
because  dod  is  not  giving  full  effect  to  its  policy  for  ot&e  at  each  decision 
point,  including  major  milestones.  On  numerous  occasions  we  have 
reported  the  negative  effects  of  systems  that  have  been  produced  with¬ 
out  adequate  ot&e.  The  barriers  to  earlier  ot&e  can  be  overcome  with 
better  planning. 

To  strengthen  dod’s  recent  reaffirmation  of  the  proper  role  of  ot&e  as  a 
key  internal  control  in  the  systems  acquisition  process  made  through  the 
Defense  Management  Review  initiatives,  we  recommend  you  ensure  that 
dod  acquisition  and  testing  directives  currently  under  revision 

clearly  establish  the  need  for  ot&e  before  production  start-up; 
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•  define  when  or&E  must  occur  and  when  it  may  be  appropriate  for  deci¬ 
sionmakers  to  rely  on  operational  assessments  in  lieu  of  or&E  for  produc¬ 
tion  start-up  decisions,  as  previously  recommended  in  our  May  1989 
report; 

•  require  the  services  to  plan  for  and  conduct  earlier  or&E;  and 

•  ensure  that  acquisition  strategies  for  major  weapon  systems  require  sys¬ 
tem  or  subsystem  prototypes  be  built  where  practical  and  that  these 
prototypes  be  operationally  tested  before  production  start-up. 


Scope  and 
Methodology 


To  determine  whether  perceived  barriers  precluded  early  crr&E,  we  per¬ 
formed  work  at  test  and  program  oversight  offices  within  OSD,  military 
service  headquarters,  and  operational  test  agencies,  and  applicable  pro¬ 
gram  executive  and  management  offices.  On  the  basis  of  a  judgment 
sample  drawn  with  direct  osd  and  service  input,  we  examined  six  cur¬ 
rent  systems  and  reviewed  our  prior  and  recently  reported  work  on 
additional  systems.  (See  apps.  I,  II,  and  III.)  We  also  performed  an  exten¬ 
sive  literature  search  and  review  of  available  documents  on  defense 
acquisition,  testing,  prototyping,  and  concurrency. 

We  did  not  obtain  written  agency  comments  on  this  report.  However,  we 
discussed  the  information  in  this  report  with  dod  officials  and  incorpo¬ 
rated  their  comments  as  appropriate,  dod  officials  were  very  receptive 
to  our  findings  and  conclusions  and  believed  that  systems  acquisition 
changes  would  be  helpful  to  the  ot&e  community.  They  pointed  out  that 
Defense  Management  Review  initiatives  and  the  Under  Secretary  of 
Defense  for  Acquisition  are  stressing  the  need  for  earlier  or&E. 

We  conducted  our  work  from  August  1989  through  February  1990  in 
accordance  with  generally  accepted  government  auditing  standards. 


As  you  know,  31  U.S.C.  720  requires  the  head  of  a  federal  agency  to 
submit  a  written  statement  on  actions  taken  on  our  recommendations  to 
the  House  Committee  on  Government  Operations  and  the  Senate  Com¬ 
mittee  on  Governmental  Affairs  not  later  than  60  days  after  the  date  of 
the  report.  A  written  statement  must  also  be  submitted  to  the  House  and 
Senate  Committees  on  Appropriations  with  the  agency’s  first  request  for 
appropriations  made  more  than  60  days  after  the  date  of  the  report. 

We  are  sending  copies  of  this  report  to  the  Chairmen,  House  Committees 
on  Government  Operations  and  on  Armed  Services  and  Senate  Commit¬ 
tees  on  Governmental  Affairs  and  on  Armed  Services;  the  Secretaries  of 
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the  Army,  Air  Force,  and  Navy;  the  Director,  ot&e;  the  Deputy  Director 
for  Defense  Research  and  Engineering/Test  and  Evaluation;  the  Direc¬ 
tor,  Office  of  Management  and  Budget;  and  the  military  service  opera¬ 
tional  test  agencies.  We  will  also  make  copies  available  to  other 
interested  parties  upon  request. 

This  report  was  prepared  under  the  direction  of  Mr.  Paul  F.  Math,  Direc¬ 
tor,  Research,  Development,  Acquisition,  and  Procurement  Issues,  who 
may  be  reached  on  (202)  275-8400  if  you  or  your  staff  have  questions 
concerning  the  report.  Major  contributors  to  this  report  are  listed  in 
appendix  IV. 


Sincerely  yours, 


Assistant  Comptroller  General 
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OT&E  Schedules  for  Six  Weapon  Systems 
We  Reviewed 


Fiscal  years 

Service/system 

Program  start 

Planned 

OT&E 

Initial 

production 

decision 

Source  of  OT&E  test 
articles* 

Army 

Lme-of  Sight-Forward-Heavy  (Air  Defense  Anti- 
Tank  System) 

1987 

1990 

1987n 

Low-rate  initial  production 

Non-Lme-of-Sightc  (Fiber  Optic  Guided  Missile) 

1987 

1994 

1 99 1 h 

Full-scale  development 

Air  Force 

Advanced  Tactical  Fighter0 

1981 

1997 

1994° 

Full-scale  development 

Short-Range  Attack  Mlssile-ll0 

1985 

1992 

1991° 

Full-scale  development 

Navy 

Long-Range  Anti-Submarine  Warfare  Advanced 
Capability  Aircraft0  (P-7A) 

1986 

1992 

1 992D 

Full-scale  development 

Fixed  Distributed  System0 

1984 

I988d 

1989h 

Demonstration/validation 

aWithout  extensively  reviewing  additional  systems,  we  noted  that  several  others  used  full-scale  develop¬ 
ment  items  for  producing  OT&E  test  articles  These  were  MK-48  Advanced  Capability  Torpedo. 
Extremely  Low  Frequency  Communication  System.  AN-SQS  53C  Long-Range  Sonar,  and  Airborne  Sell- 
Protection  Jammer 

bFor  low-rate  initial  production 

Prototypes  have  been  built  or  planned  for  these  systems  However,  the  services  are  not  planning  to 
use  the  Non-Line-of-SIght  Missile.  Short-Range  Attack  Missile-ll,  and  Advanced  Tactical  Fighter  proto¬ 
types  for  OT&E 

dAn  OT&E.  such  as  the  one  described  in  footnote  3  on  page  2,  was  successfully  conducted 
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Examples  of  Systems  Noted  in  Our  Recent 
Reports  That  Did  Not  Have  OT&E  Planned 
Before  Production  Start-Up 


Abrams  Tank:  Block  II  Modifications  Not  Ready  for  Production 
(nsiad-90-57,  Feb.  28,  1990) 

•  Army  will  commit  $166.4  million  to  the  M1A2  tank  program  before 
development  prototype  testing  is  begun. 

•  oso  nad  not  approved  the  Army's  test  and  evaluation  master  plan  as  of 
August  1989  because  it  was  concerned  that  the  Army  had  not  suffi¬ 
ciently  planned  live-fire  testing  and  or&E,  which  needs  to  be  completed 
and  evaluated  before  the  production  decision  is  made  in  August  1991. 

Strategic  Bombers:  B-2  Program  Status  and  Current  Issues  (nslad-90- 
120,  Feb.  22,  1990) 

•  B-2  aircraft  acquisition  strategy  includes  ordering  a  large  number  of 
planes  before  the  necessary  testing  demonstrate  that  the  B-2  can  per¬ 
form  its  mission  is  completed. 

1CBM  Modernization:  Rail  Garrison  Production  Decision  and  Launch  Car 
Acquisition  Should  Be  Delayed  (nsiad  90-19,  Dec.  7,  1989) 

•  No  ot&e  of  the  complete  weapon  system  (missiles  and  rail  cars)  wi  l  have 
been  conducted  prior  to  the  initial  production  decision. 

•  We  recommended  that  the  Secretary  of  Defense  delay  the  April  199' 
initial  Rail  Garrison  production  decision  until  the  Air  Force  has  con¬ 
ducted  some  ot&e  of  the  complete  system. 

Navy  Weapons  Testing:  Defense  Policy  on  Early  Operational  Testing 
(NSIAD-89-98,  May  8,  1989) 

The  following  seven  programs  were  authorized  for  low-rate  initial  pro¬ 
duction  with  no  CT&E: 

•  ALQ-165  Airborne  Self-Protection  Jammer. 

•  SQS-53C  Sonar. 

•  E-6A  Aircraft. 

•  MK-45  Capsule  Launching  System. 

•  Ocean  Surveillance  Information  System  Baseline  Upgrade. 

•  TB-23  Accelerated  Thinline  Towed  Array. 

•  AN/BSV-2  Submarine  Combat  System. 
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Examples  of  Systems  Noted  in  Our  Recent 
Reports  That  Did  Not  Have  OT&E  Planned 
Before  Production  Start-Up 


Navy /Air  Force  Still  Developing  Separate,  Costly  Radar  Warning  Receiv¬ 
ers  (nsiad-87-167,  July  1,  1987) 

Five  radar  warning  receivers  (rwr)  started  production  before  or&K  was 
completed. 

Starting  production  before  adequately  testing  the  systems  has  resulted 
in  the  purchase  of  equipment  that  cannot  be  used  for  its  intended  pur¬ 
pose,  production  of  rwr  that  were  placed  in  storage  rather  than 
installed,  and  deployment  of  rwr  judged  operationally  unsuitable  to  U.S. 
combat  forces  by  testing  officials. 
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Examples  of  Reports  Illustrating  Where 
Production  Decisions  Were  Made  Before  OT&E 
Was  Started  or  Completed 


Operational  Test  and  Evaluation  Can  Contribute  More  to  Decisionmak¬ 
ing  (N’SIAD-87-57,  Dec.  23,  1986) 

Our  analysis  showed  41  cases  where  production  was  approved  before 
ot&e  was  started  or  completed.  Thirty-one  of  the  41  cases  were  identi¬ 
fied  in  our  reports  and  summarized  in  this  December  23,  1986,  report. 
The  31  reports  are  listed  below. 

1.  Adverse  Effects  of  Large-Scale  Production  of  Major  Weapons  Before 
Completion  of  Development  and  Testing  (B- 163058,  Nov.  19,  1970) 

2.  The  Importance  of  Testing  and  Evaluation  in  the  Acquisition  Process 
for  Major  Weapon  Systems  (B-163058,  Aug.  7, 1972) 

3.  Review  of  the  Adequacy  of  Department  of  Defense  Test  Resources 
(psad-75-84,  Apr.  30,  1975) 

4.  Effectiveness  of  Testing  of  Selected  Major  Weapon  Systems 
(psad-75-74,  June  4,  1975) 

5.  Need  for  Additional  Test  and  Evaluation  on  the  Major  Caliber  Light¬ 
weight  Gun  (PSAD-77-4,  Nov.  5,  1976) 

6.  Navy  Operational  Test  and  Evaluation  -  A  Valuable  Tool  Not  Fully 
Utilized  (psad-78-77,  Mar.  29, 1978) 


7.  Operational  Testing  of  Air  Force  Systems  Requires  Several  Improve¬ 
ments  (psad-78-102,  June  2,  1978) 


8.  Department  of  Defense's  Conduct  of  Operational  Test  and  Evaluation 
of  Foreign  Built  Weapon  Systems  (psad-79-131,  July  25,  1978) 

9.  Practices  and  Procedures  for  Follow-On  Operational  Testing  and  Eval- 
uation  of  Weapon  Systems  by  the  Military  Services  (psad-79-i,  Oct.  19, 
1978) 

10.  Army  Operational  Test  and  Evaluation  Needs  Improvements 

(psad-80-2,  Nov.  13,  1979) 

11.  Ml  Tank’s  Reliability  is  Still  Uncertain  (psa D-8Q-20,  J an.  29,  1980) 

12.  F/A-18  Naval  Strike  Fighter:  Its  Effectiveness  Is  Uncertain 
(psad-80-24,  Feb.  14,  1980) 
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Examples  of  Reports  Illustrating  Where 
Production  Decisions  Were  Made  Before 
OT&E  Was  Started  or  Completed 


13.  Cruise  Missiles:  Status  and  Issues  as  They  Near  Production 
(psad-80-19,  Feb.  28,  1980) 

14.  Future  Procurements  of  Army’s  Copperhead  Projectile  Should  Be 
Contingent  on  Improvements  in  Performance  and  Reliability  (psad-8i-4, 
Nov.  13,  1980) 

15.  Review  of  the  High  Speed  Anti-Radiation  Missile  Program 
(masad-81-7,  Feb.  28,  1981) 

16.  Some  Land  Attack  Cruise  Missiles  Acquisition  Programs  Need  to  Be 
Slowed  Down  (masad-81-9,  Feb.  28,  1981) 

17.  Most  Critical  Testing  Still  Lies  Ahead  for  Missiles  in  Theater  Nuclear 
Modernization  (masad-81-15,  Mar.  2,  1981) 

18.  The  Army’s  Advance  Attack  Helicopter  Is  Not  Ready  for  Production 
(MASAD-82-8,  Dec.  1,  1981) 

19.  Air  Launched  Cruise  Missile  Shows  Promise  but  Problems  Could 
Result  in  Operational  Limitations  (C-masad-82-13,  Feb.  26,  1982) 

20.  Progress  of  the  Light  Armored  Vehicle  Program  Should  Be  Closely 
Monitored  (masad-82-41,  Aug.  10,  1982) 

21.  Results  of  Production  Testing  Should  Be  Considered  Before  Increas¬ 
ing  Patriot’s  Production  (masad-83-7,  Jan.  26,  1983) 

22.  Acquisition  of  the  Over-The-Horizon  Backscatter  Radar  System 
Should  Be  Reevaluated  (masad-83-14,  Mar.  15,  1983) 

23.  The  B-l  Bomber  Program  -  A  New  Start  (masad-83-21,  Apr.  13,  1983) 

24.  Better  Planning  and  Management  of  Threat  Simulators  and  Aerial 
Targets  Is  Crucial  to  Effective  Weapon  Systems  Performance 
(masad-83-27,  June  23,  1983) 

25.  Air  Force  and  Navy  Trainer  Aircraft  Acquisition  Programs 
(masad-83-22,  July  5,  1983) 

26.  Results  of  Forthcoming  Critical  Tests  Are  Needed  to  Confirm  Army 
Remotely  Piloted  Vehicle’s  Readiness  for  Production  (nsiad-84-72,  Apr.  4, 
1984) 
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Examples  of  Reports  Illustrating  Where 
Production  Decisions  Were  Made  Before 
GT&E  Was  Started  or  Completed 


27.  Status  of  the  Peacekeeper  (MX)  Weapon  System  (nsiad-84-112,  May  9, 
1984) 

28.  Army’s  Decision  to  Begin  Production  of  the  High  Mobility  Multipur¬ 
pose  Wheeled  Vehicle  Was  Premature  (nsiad-84-136,  June  12,  1984) 

29.  Army  Has  the  Opportunity  to  Recompete  DAS3  Purchases  and 
Improve  Automated  Battlefield  Support  Qmtec-84-20,  Sept.  28,  1984) 

30.  Production  of  Some  Major  Weapon  Systems  Began  With  Only  Limited 
Operational  Test  and  Evaluation  Results  (nsiad-85-c>8,  June  19,  1985) 

31.  Evaluation  of  Army’s  Mobile  Subscriber  Equipment  Program 
(nsiad-85-117,  July  16,  1985) 


i 


! 
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Major  Contributors  to  This  Report 


National  Security  and 
International  Affairs 
Division,  Washington, 
D.C. 


Michael  E.  Motley,  Associate  Director 
Lester  C.  Farrington,  Assistant  Director 
Maureen  Machisak-Herndon,  Evaluator 


Norfolk  Regional 
Office 


Fred  Harrison,  Regional  Management  Representative 

Leslie  Gregor,  Evaluator-in-Charge 

Clifton  Spruill,  Site  Senior 

Julie  Chapman,  Evaluator 

Jim  Marshall,  Advisor 


Denver  Regional 
Office 


Ted  Baird,  Regional  Management  Representative 
Ernest  Beran,  Regional  Assignment  Manager 


(3M231) 
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